Facial micro-expression(ME) recognition has great significance for the progress of human society and could find person's true feelings. Meanwhile, ME recognition faces a huge challenge, since it is difficult to detect and easy to be disturbed by the environment. In this paper, we propose two novel preprocessing methods based on Pixel Residual Sum. These methods can preprocess video clips according to the unit pixel displacement of images, resist environmental interference and be easy to extract subtle facial feature. Furthermore, we propose a Cropped Gaussian Pyramid with Overlapping(CGPO) module, which divides images of different resolutions through Gaussian pyramids and crops different resolutions image into multiple overlapping subplot. Then, we use a convolutional network of progressively increasing channels based on the depthwise convolution to extract preliminary features. Finally, we fuse preliminary features and make position embedding to get last features. Our experiments show that the proposed methods and model have better performance than the well-known methods.